Abstract— Fre—-ICT advancements—in information and _ecoenmmuncation 
technelegies-GCF)-and wireless sensor netwerks—-network advancements 
have played a pivotal role in the design and development of real-time 
healthcare solutions for community-dwelling older adults living in health- 
assistive smart homes. These smart homes are equipped with sensor 
technologies that can detect and record everyday actvitiesatsetnewn 
as—Aactivities of Daitdaily Living—living (ADLs), and provide clinically 
relevant information to eHnicians—healthcare providers and other 
stakeholders by remotely whe—are—monitoring the health, wellness, and 
safety of the-smart home residents-remetely. As a result, older adults can 
age independently, safely, and comfortably in their living environments. 
Many ef+he-smart home approaches are limited to the monitoring of a 
single user in a home—, but there are many Hewever—there—are—certain 
scenarios of multiple occupancy (i.e., where-where more than one person 
lives in the home, for assistance or companionship). Attempts have been 
made to model ADELs—ADL knowledge for a multi-user environment, 
hewever—but they are limited to modeling atomic and complex activities 
such as washing dishes or making a meal, and fails—fail to capture 
composite activities such as two people engaged in making a_meal and 
washing dishes at—the—same—simultaneously, time—by—twe—persens 
functioning in parallel or collaborating in certain tasksand meving-atable 
by—_twe—persens—coltaberatively. Additionally, they fail to model and 
reason with expert (e.g., eHhnictans—clinician) knowledge,—texperts 
knewledge}—_when underlying information for decision support is 
incomplete and/ or contradictory. Moreover, very few attempts have been 
made to relate ADL—activitiesADLs with health events, such as to 
recognize cognitive impairment (e.g., Alzheimer’s, a person with 
advancing cognitive decline, walking, or ruminating in the same room). 
This survey paper discusses various techniques used for performing 
ADLs in the sensor-based environment by single-user and multi-user. 
Based on composite activities, their temporal nature, and uncertainty 
handling, this paper can—differentiate—analyzes techniques used in the 
previous Hterature—studies, techniques—discusses research challenges, 
and provides future directions. 


I. INTRODUCTION 
fheaAdvancements sin fen ncton and cop eationsecneclocies 
{ICT} [1] and wireless sensor networks [2] have led to the design and 
development of real-time information systems for decision making in 
severalt-various domains sueh—as—of_ e-commerce [3], e-education [4], 
manufacturing [5], security and surveillance [6], disaster management 


[7], -and manymereother fields. In hHealthcare-+s-net-an-exceptionancd, 
the development of ubiquitous environments for real-time patient 


monitoring has significantly increased the quality of care and life for 
patients. 


The World Health Organization-WHO) mentiens—reported that the 
number of people aged over 60 will increase from 900 to 2000 million 
that-betweern from 2015 asd-io 2050—4e sector ofthe pepsilatier ats 
AAere—ran—_60—years—eHliaecrease—_rena 90016 2060—nttern—_ [8]. 
AdditionaltyIn Australia, ~aeceerding—te—the Australian Law Reform 
Commission— projects that the population aged 65 or over wHłwill be 
projected-_be 22.6% by 2054-55, which-up from is-15% ofthe pepulation 
in 2014-15 [9]. Heweverlhe increasing number and proportion of older 
people in society, contemporaneous with increasingly advanced 


technological solutions, means that smart technologies are likely to play 
an increasing part in improving the quality of life of older people-. #-It is 


possible to monitor Communitycommunity-dwelling older adults—_who 
suffer from chronic physical condition and/or cognitive changes such as 
dementia, and changes in health could be detected early using smart 
home sensors. Fhis-Such sensing infrastructure provides low--cost, low- 
power consumption, and high-performance solutions—anc—, and it has 
been estimated that smart homes wilt-can save centribute-about $USD 7 
trillion eest—reductien—in costs associated with healthcare annually 
worldwide [10]. Moreover, this technology may support health 
professionals (i.e., physicians, -decters—nurses, or caregivers) to better 
allocate resources in case of emergencies. 


The real-time healthcare information system identifies and records the 
activities of daily living (ADLSs) of patients in their homes {24+4/224-/[21], 
[22]. The concepts of ese was first expounded senn Aeres ehehe 

n in the 1950s by Sidney 
Katz [13], who developed the Katz Index of Independence in Activities of 
Daily Living, also known as the Katz ADL, which is mainly oriented 


toward individual activities performed by individuals. However, for real- 


world smart technology scenarios, of use in the emerging paradigm of 
smart home monitoring to facilitate the lives of older people, the 
traditional atomistic ADL oo is lacking, _ as individuals can Wa 


tee Ss multiple e E EA e a 
(composite activity), or can co-habit and share in tasks with others 
(collaborative activity). 


To create a hierarchical/classified structure for composite activity, we 
are adopting the activity theory [14] as a foundation for knowledge 
development. In this regard, we consider the actions as an atomic activity 
such as walking, standing, eating, mobility [15], are performed by the 
individuals at a particular location, at a particular time, with certain 
objects [16], [17]. A combination of atomic activities, such as making 


cooking a dinner, er—-making—a—meatis called a complex activity. The 


combination of two or more complex activities is called a composite 
actites—activity, such as making a meal and washing dishes. These 
activities can use temporal or spatial information to specify inter-activity 
relationships [17]. Furthermore, +he-researchers have investigated and 
modeled different types of activities (atomic, complex, and composite) 
with respect to time, either as sequential, interleaved, and concurrent for 
single--person environments;;_and parallel and collaborative activities for 
multi-user environments [18]. 


Significant research has been done for ADLs identification, known as 
Activity—activity Reeegnitien—recognition (AR) [19], [20]. This research 
has been conducted to model simple and composite ADLs for a single- 
individual environment [17]. The researchers modeled contextual 
information using ontologies [11], [12], [21]-[23] and machine learning 
approaches to perform probabilistic reasoning in the semantic web to 
detect simple or complex activities [24], [25] and anomalies in activities 
by using neural network [26]. Some studies found that that event can be 
captured in data streams and that complex event processing and 
reasoning can be used to predict complex and composite activities [27]- 
[29]. Some researchers took into consideration the uncertainty of the 
data/conflicting preference, to predict complex activities using Answer 
Set Programming language [30], [31]. 


The literature shows that there is a greater interest in a multi-user 

environment focusing on ADL-activitiesADLs where a composite activity 
has been modeled. However, few researchers only track and identify 
multi-users in the smart home environment using machine learning 
algorithms [32], [33], by-using-machinetearning-algerithms-while others 
recognize activities individually, —and—cooperatively, and in parallel by 
using Bayesian Networks (BN), Recurrent Neural Network (RNN), 
Conditional Random Field (CRF), and Hidden Markov Models (HMM) 
techniques [34]-[37], and related consumer mobile applications have 
already been developed [38]. 


Il. LITERATURE RREVIEW 


Many studies have been conducted for real-time sensor-based activity 
modeling and reasoning of activities using a range of techniques, 
frameworks, and methodologies. After a thorough literature review, the 
current approaches can be classified into three main categories name}: 
learning-based-appreaches, Knewledgeknowledge-based-appreaches, and 
hybrid approaches, as shown in Figure 1. »+The following sections—we 
wit discuss each category in detail. 
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Fig. 1. Activity Recognition Methods 


A. Learning-Bbhased appreaches-Approaches 

+he—tLearning-based data-Driven—driven 
Appreaches-approaches (DDA) _are based on 
machine learning (ML) techniques—These—-metheds—, by which develep 
models are developed based on pre-existing datasets rather than 
intuition or person experience. The benefit of learning-based methods is 
the ability to handle uncertainty. Some techniques—e-g, such as = 
recurrent neural network (RNN), -also deal with temporal information 
[12]. In learning-based sega anro can generally be 


generated in four ways, explained i detail below: 
Supervisedsupervised, Oa semi- S NT and 


reinforcement—lr—hciolowing sectors mrem ciscuss-each—ot+-he wip 
fen, 


1)Supervised ML Methedsmethods 

Most ef—-the—learning-based approaches use supervised machine 
learning algorithms, which require a pre-existent labeled dataset of a 
users—user behavior to infer the activity model. In these methods, the 
training data is used to train and test the model for accuracy and 
precision [23]. Once the model is trained, sensor data is feed into it to 
detect physical activities, such as gait speed, postures, and motions, and 
Heatth-health events. Supervised ML methods include Bayesian networks 
(BNs) [39], partially observable Markov decision process (POMDP) [9], 
Nnaive—ive Bayes [40], C4.5 decision tree [41], Suppert-support vector 
machine (SVM), Markov Ghaimschains, Hidden—hidden Markov Medels 


models (HMMs}-) [27], [42], Legistie_logistic Regressien_regression [40], 
the-conditional random field (CRFs) [27], [43], [44], and neural networks 


(NNs) [26], [45]-are-some-examples-efsupervised Mhmetheods. 


By using supervised learning, most ef—the—systems have been 
dedicated to an activity recognition environment for single individuals 


[10], [46], while few researchers are modeling activities in a multi-user 
environment. A benchmark study for multi-resident smart homes was 
presented by Tran et al. {84+-[34], in Im this-studywhich the researchers 
learned about the effectiveness of temporal learning algorithms and non- 
temporal learning by using sequential data and temporally manipulated 
features, respectively. ADL recognition is—-was also performed by [36]- 
[38], [47], in a multi-resident smart home based on Bayesian-nebwerks 
{BN_}-Hidden-Markeyvmedel (and HMM} techniques. Chen et al. [35] 
proposed a system to recognize complex activities for the multi-resident 
environment by using both techniques-suchas-HMM and CRF. 


2)Unsupervised methods 

Due to some limitations in the supervised approach, studies have 
investigated unsupervised approaches [48]-[50] which generate activity 
models that rely on a training set of unlabeled sensor data. Unsupervised 
methods are divided into twe—+pes—Cclustering and Association 
association types [51], [52]. Most efrelated research the—researehers 
used unsupervised clustering ML methods in a-single-user environments, 
to detect various physical activities, such as walking, sitting, 
Standingstanding, and jogging. For example, Lu et al—. [49] provided a 
method for ADL classification by using a smartphone accelerometer. 
Negin et al. [48] proposed a framework for early diagnosis of cognitive 
impairments and ADLs activity discovery and scene modeling in health 
care. In one study, Whiethe location of multi-ple users in a home is-was 
detected by the vectorization approach using an algorithm named sMRT 
to track the location of each resident-b}—, using the clustering technique, 
and the system could estimates-estimate the number of residents in the 
smart home with the help of event association [33]. 


3)Semi-supervised methods 

Semi-supervised learning methods improve the model computed 
through the training set by using unlabelled data [25]. In semi-supervised 
learning, both labeled and unlabelled training data ef traimimng-—are used. 
In other words, it is a contrast of supervised learning (labeled data) and 
unsupervised learning (unlabelled data). There are some examples of 
semi-supervised methods i.e. self-training, co-training, graph-based 
methods, expectation-Maximizaten—maximization (EM) with generative 
mixture models, and transductive suppert-vecter-machinesSVMs— [53]. 
Fhese—Such learning-based methods can be atse—transductive or 
inductive. Various researchers proposed algorithms and frameworks in 
the field of activity recognition by using semi-supervised_ methods, 
whereby available unlabeled samples were utilized to enhance the 
performance of activity learning and activity recognition when compared 
with standard supervised techniques[54]-[57]have-prepesed_algerithms 
anc trameworks—+i—tse—Helt—ol —aebiviiy—_receqniiien—by—using seme 
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when-comparedacithstaacdardsiuperdsectechiiaiues, However, in these 
systems, there is a need for human annotator’s effortsis+equired, which 
se-usualtlimits the number of labeled samples-stimited. 


4)Reinforcement: 

Reinforcement learning (RL)—is—, also known as _ neurodynamic 
programming, is an approach for building agents automatically, whereby 
andthese agents take a reward function after performance measures. 
The system can direct its problem space and provides feedback in terms 
of rewards and punishments [58]. For example, Yamagata et al. [59] used 
the—an RL algorithm to address multiple sources of feedback to 
accelerate ¢he-learning speed, and they achieves-achieved a good trade- 
off between wearable power consumption and activity recognition 
accuracy. In _ activity recognition, Richard [60] proposed a 
recommendation system to guide Alzheimer’s patients +»—performing 
their activities-of daily ving (ADLs} ciagnesedowith Alzheimer's disease, 
based on changing human mental state, behavior, and environmental 
contexts. Whereas—Zhang et al. [61] proposed an effective way of 
recognizing human behavior activities in smart homes by using Peep 
deep reinforcement Learning-learning (DL). Table 2I]— Shews-shows the 
various learning-based ML methods used in Si single- and 
Multimulti-User—user environments for Atemie—atomic and Complex 
complex activities. The learning-based ML approaches are black-box 
models, they—which do not provide the direct understanding of 
mechanism by which a model work in terms of transparency. For 
example, most of the systems do not provide explanation/ justification of 
outcomes that are critical in healthcare related problem for 
explanation/understanding of the patient, whereby clinician/experts_and 
other experts require needs—more information fer-to supperting-support 
their diagnesisdiagnoses. 


FABLE-TABLE I. Learning-Based Approaches 


Single-user Technique ADLs Granularity Uncertainty Data Set 
{Multi-user Temporal level of ADL | handled 
relation 
Single-User Supervised Sequential Simple Yes CASAS Dataset 
activities (Fuzzy Set) 


Pn) SPE Single-User Supervised Sequential Simple Yes Aruba and CASAS 
[38] (CNNs) 

lo On TE SB) Single-User Reinforcement Sequential Simple Yes Smartlab 

[74] learning Learning 

Automata (LA) 

LAS Ae TE PE Single-User Supervised Concurrent Complex No House_n project 


N 
oO 


PE Sa ee Single-User Supervised sequential, Atomic, No UCI, EU project, and 
concurrent Complex caU 


Single-User Unsupervised Sequential, Simple, No No 
Interleaved, and Complex 
Concurrent 


Wang CEPI Multi-User Unsupervised Joint Events Atomic No TMO04 and the 
[45] Kyoto 

Gu COLET E= Multi-User Supervised parallel Complex No CASAS Dataset 

al., [62] {cooperative 

activities. 

Roy et al., [50] Multi-User Supervised Parallel Complex No a.k.a PUCK dataset 
fen eee ese Multi-User Supervised Sequential, Complex No CASAS Dataset 
Parallel 

fe eee Multi-User Supervised Sequential, Complex No CASAS Dataset 
[79] Cooperative, 

Collaborative 


B. Knowledge-Based Approaches 


In the knowledge-based approach (KDA), “an activity model is 
developed through the incorporation of rich prior domain knowledge 
obtained from the application domain, using knowledge engineering and 
knowledge management techniques” [23]. In t:e-KDA-appreaches, the 
activities can be modeled and expressed manually through numerous 
formats (i.e., rules, logical axioms, and ontologies [12], [21]-), and can be 
mined from other sources (e.g., Web resources, or unlabelled datasets of 
activities} [25]), that-which can be interpreted by humans and machines, 
rather than ef—the acquisition of labeled training data sets. The 
knowledge structure of KDA is modeled by—using various techniques, 
mainly classified as Menetenie—monotonic and Nesnon- 
Meneoteniemonotonic. 


1)Monotonic logic 

Monotonic logic is also known as standard logic or classical logic, 
whereby once something is true, it is true forever [62]. It is used as a 
specification language to represent declarative knowledge. However, it 
represents monotonicity property based on the conclusion involved by a 
body of knowledge;;_if additional knowledge is added there is no effect 
on the conclusion, it remains valid. Monotonicity performs its role in the 
design of an intelligent agent reasoning in dynamic domain context, such 
as the performance of complex APL-acthitiesADLs [62]. The monotonic 
logic is further divided into two categories based on the source of 
information being modeled. 


e Information extraction from existing sources__(mining-based 


approach) 


Mining-Based Appreach:Different studies have been conducted using 
a mining-based approach [19], [63], [64]. These approaches retrieve 


definitions of activities and phrases that define involved objects and the 
activity performance process by using information retrieval techniques. 
The system used this approach is designated to overcome the deficiency 
of annotated ADL data problems and the variation or complex activity 
performance between various individuals [12]. To extracts—extract 
temporal information to compute more accurate activity models, by 
mining—activity data can be mined from pictures and videos, in which 
context visual media has been efficiently used to mine activity models, 
including if sporadic activities are involved deneby([65]-and result shows 
Ree esl ecie ee eat bh teed eae ocol also —t 
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However, feature engineering is key to the development of data 
mining applications [23]. An ambient assistant system, named; ontology- 
based Ambient-Aware LifeStyle tutoring for A BETter Health 
(ontoAALISABETH) [66], -provides an ambient assisted living framework 
to monitor the lifestyle of general older people— (who are not suffering 
from major chronic diseases or severe disabilities). It integrates an 
ontology with rule-based and a complex event processing (CEP) engine 
forto supperting-support thetimed reasoning, but it cannot teaekte-deal 
with conflicting events. 


e Information modeling 

In the logical-based approach, to present the knowledge about activity, 
different types of logical formalisms are used [12]. In this regard, various 
methods were used to build a recognition model that infer actions or 
activity intentions through general knowledge-based rules, whereby 
training data was-is not required. Related applications have been used in 
such-as-—a method to identify human intention based on percept sequence 
by using event calculus [67], and tagged visual contents shared on the 
web [68]. In healthcare, the iKnow activity recognition system H+}for a 
single-user environment provides capabilities—which— that enhance the 
extraction of behavior patterns and behavior changes, including how 
activities are performed, with idiosyncratic—_and habitual knowledge 


AL]: 


Triboan et al. [43] developed a multithreaded decision algorithm and a 
system prototype capturing generic knowledge and inhabitant-specific 
preferences for conducting composite ADLs to support the segmentation 
process—, while Another researcher —Bennasar et al. [12] proposed a 
system to support older people to live independently in their own homes 
by increasing the quality of life in terms of care, safety, and security. A 
non-quantitative approach to plan recognition was found to minimize 
uncertainty about the prediction of the observed agent’s behavior for 
people with cognitive impairments, based on lattice theory and action 
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mali: -user environment, LO ET et al. [70] proposed a 
novel approach to recognize concurrent activities, whichis-independent 
of sensor deployment and activities of interest. 


2)Non-mMonotonic Legte-logic 
Fhe—authersin {62} define-Non-monotonic logic has been defined as 
“Noen-Monotenictogicis—a kind of inference in which reasoners draw 


tentative conclusions, enabling reasoners to retract their conclusion(s) 
based on further evidence~.” [62]. In other words, facts and rules can be 


changed at any time_depending on the type of reasoning involved, as 
described below. Fre-citferentare-semeexamptles-civven below: 


a) Defeasible Reaseningreasoning: This is a non-monotonic logics 
formalism that delivers instinctive knowledge representation and 
advanced resolution mechanisms, such-including for as-inconsistent [71] 
and conflicting information (uncertainty) [72]. Defeasible reasoning is 
better than previous classical logics in terms of computational 
complexity, dealing with incomplete information, and imitiates—initiating 
new reasoning for non-accustomed users (e.g., doctors, patients, etc.). In 
this context, the ReDEF (Context-aware Recognition of Interleaved 
Activities using OWL 2 and Defeasible Reasoning) framework is—was 
proposed [72+ for ADL recognition, which—and it is being used fer—to 
supperiag-support the diagnosis of Alzheimer’s disease in a controlled 
environment for interleaved activities- [72]. 


b) Argument-Based-based Reasenmgreasoning: Argumentation theory 
is also known as “common-sense” reasoning, because a person usually 
does something and reasening—reasons about it by evaluating the 
potential results [62]. The Argumentationargumentation-based approach 
is also used to deal with conflict data, whereby preference arguments are 
built—_that they-witdefine whichone-wil be-a suitable solution, unlike in 
a rule-based system, where preferences are defined one on another for 
conflicting data. 


In this regard, Nieves et al. [73] proposed an activity a—-framework 
Deas eg ee ee eel ee pe foal esrebieec comp nes 
activity theory [74] and Dung’s [75] argumentation theory _in one 
threugh—coherent te—tool to identify the fulfillment of particular atomic 
activities through a union of fragments of activities. Folewed—by—The 
same research team also the-same—researechersin—7/4} introduced an 
argumentation approach that takes an extended logic program (ELP) as 
input, and gives a set of arguments with the respective disagreements 
(set of attacks) among them as outputs |71]. They were using two 


semantics for an ELP Answer-answer set: semantics, and well-founded 
semantics. In that scenario, they had-aimed to follow Answeranswer Set 


set Pregramming-programming roots for capturing negation as failure in 
an argumentation-based setting. 


Based on human motives, goals, and prioritized actions, Guerrero et 
al. [76], proposed an argument-based approach for tracking and 
monitoring older people’s complex activities. In this regard, ALI (Assisted 
Living System) is an argumentation-based multi-agent system [77] 
developed for providing support for behavior change in single person 
environment with mild depression and socially isolated individuals in the 
home environment. It empowers the individual with the required 
feedback or recommendations to change their habits and behavior 
through an argumentation-based approach. Table II/3— Shews-shows the 
ADL recognition performed by different researchers by using knowledge- 
based techniques such as Menetenice—monotonic and Nennon-Menetenic 
monotonic techniques for various types of activities (Simptesimple, 
complex, and composite). 


TABLE-TABLE IT. Knowledge-Based Systems 
Single-user Technique ADLs Temporal} Granularity Uncertainty Data Set 
/Multi-user relation level of ADL handled 
TT EE tia Single-User Monotonic Sequential Complex No No 
al., [22] (Ontology) STRETCH (Socio-Technical Resilience 
for Enhancing Targeted Community 
Healthcare) 
Triboan et al., Single-User Monotonic Parallel, Concurrent, Single,;Composite No No 
[51] Ontological incremental Activities 
modeling 


LE) rasa Single-User Monotonic Interleaved Situation No [94], [95] and CASAS, Synthetic 
al.,[21] ontologies Descriptor Dataset, ubicomp 

(OWL/OWL2 

SPARQL rules) 


GOLE- S Single User Monotonic Percept Sequence Complex Yes No 
[85] Logic-Based intensions (Uncertainty weight 
Event Calculus function) 
(o OTTO] Single user Monotonic Sequential Complex activities NO NO 
Project 
ey 


IVT) ria Single User Non-Monotonic Sequential, Complex activities Yes (Defeasible Dem@Care FP7 EU Project 
al., [89] Interleaved Logic) 


Experimental 


Rosero [80] Single User Non-Monotonic Activity-Goal Complex Yes No 
(Answer Set 
Programming) 
Ye tA Multi-User Monotonic Concurrent Complex No CASAS 
Stevenson, Ontology Dataset 
88 


C. Hybrid Approaches 

A hybrid approach is a combination of both appreaches-DDA and KDA 
approaches—, seeking to address some of the As—previously discussed; 
there-are-seme- drawbacks te-with these individual previeus-approaches; 
. the-The shared problems fer—of both approaches te—can_be resolved 
within the activity model- by using a hybrid approach-s-used. In DDA+he 


_ activities can be represented by probabilistic and statistical models 
[11]. These approaches also useful for dealing with noise and uncertainty 
of sensors measurements, heweverbut they ;lack the capability of 
capturing semantic relationships between sensor events and activities. 
While-—_KDA approaches capture complex semantic relationships, but 


these-appreaches-they are often too rigid to cope with handle noise and 
uncertainty [78]. 


In the context of a single-user environment, hybrid approaches have 
been developed which take advantage of the main benefits provided by 
unsupervised techniques, combined with ontologies for activity 
recognition. The initial model is grown by combining KDA expressivity 
power and DDA uncertainty handling; In _im—this regard, some 
researchers [24], [25], [50], [79] used a combination of ontology and 
probabilistic reasoning (i.e.. Markov Legic—logic network) to recognize 


sequential, concurrent, and interleaved activities. 


To recognize complex ADLs in a multi-resident environment, a hybrid 
approach was modeled by Roy et al. [38]. In this model two extremes of 
sensors—+e— (ambient sensers—and body-worn sensors) were used to 
exploit the users’ spatiotemporal behavior and location. Nguyen et al. 
[80] also provided a mechanism for activity segmentation and recognition 
in the context of multi-resident homes, which used ontology and 
unsupervised machine tearning ML strategies, such as pattern discovery. 
However, Mohmed [81] proposed another promising technology-such-, as 
the Fuzzy Finite State Machine (FFSM), to handle uncertainty associated 
with human behavior. Table _IV—4— explains the different hybrid 
techniques (i.e., probabilistic ontology modeled for activity recognition in 
Sequentialsequential, Mterleavedinterleaved, and GCeneurrentconcurrent 
scenarios mannerfor complex and composite activities). 


FABLE-TABLE III. Hybrid appreaches-Approaches 


Single-user | Technique ADLs Granularity | Uncertainty Data Set 
/Multi-user temporal level of ADL | handling 
relation 


fe) <a a Single-User Ontology Sequential, Composite 
[28] +Temporal Concurrent 
Formalism 


fens TE a Single-User Ontology + MLN Sequential and Complex Yes CASAS 
al., [37] Interleaved (Markov logic 
Network) 
Gayathri TS) Single-User Probabilistic Sequential, Composite Yes CASAS 
al., [36] ontology Interleaved, (Markov logic 
Concurrent Network) 


tate T. E= Single user LSTM- FFSM and Sequential Complex Yes CASAS 
al., [76] CNN-FFSM activities Fuzzy Finite 

State Machine 

(FFSM) 


Or Er M ee Single-User Probabilistic Sequential Complex Yes CASAS and 
[98] Ontology (Dempster IELAB 
Shafer theory) datasets 


Nguyen a) Multi-user Ontology + Concurrent Complex No CASAS 
al., [99] Unsupervised 
ML 


III. DISCUSSION 


A copious amount of progress has been made for modeling, reasoning, 
and prediction of Activities-of DaityLiving(ADLs}_of persens—people 
living in their homes [11], [12]. Among various approaches investigated, 
few attempts such as MLNs, ontologies, argumentation, and CEP engine 
are-offer promising techniques which try to reeegnizing—recognize ADLs 
in a single--inhabitant environment, with simple or complex activities. 
This survey paper discusses existing techniques for modeling and 
reasoning ADLs in an activity recognition environment and provides a 
taxonomy for their classification, which—that—_includes learning-based 
methods, knowledge-driven approaches, and hybrid approaches. Table I- 
shows that—_ most of the learning-based methods are used to generate 
models for modeling atomic/simple and complex APL-actaitiesADLs in 
both single- and multi-user environments. Some of these approaches 
used for ADL modeling in > Bo environment can handle 
uncertainty using fuzzy sets learning automata f{8$+}160], and 
Neural Networks [81], Sr a eee an ee 


environment can _hande tneertaintyusing fizzy sets, tearning-attomata, 
and—_Neural Netweorks—respectively—; however, they are limited to 


modeling only sequential activities. Modeling approaches for ADL in a 
multi-user environment can model complex activities, hewever—but they 
fail to model frsth++he-composite activities, and secend-—they lack the 
capability of handling uncertainty for ADL-acthitiesADLs modeling and 
predication when underlying information is incomplete and/or 
contradictory. Although limited systems can identify health events for 
ADLs, the modeling is limited to single-user contexts. 


Table II— shows that most of the monotonic logic-driven knowledge- 
based systems can model simple and complex models of ADLs activities 
in a single-user environment; only one approach could model composite 
ecliviiios-except— [43 ]-appecech—bat-casnsaedel_compesite—acticties, 


HeweverWhile —they can handle incomplete information using weights, 
butthey cannot process aet-contradictory data, as they are not able to 
deal with uncertainty_[67]. Additionally, a non-monotonic logic-based 
system for the single-user environment can model and reason about 
sequential and complex activities and perform support modeling and 
reasoning with incomplete and contradictory knowledge, by using 
defeasible logic and answer set programming. However, these systems 
are unable to model composite activities and they use a single criterion 
to resolve the contradictory knowledge. In a multi-user environment, one 


study modeled complex activities by applying monotonic logic, but the 
system was unable to model composite activities, and could not handle 


Aet-ef-werkis—Major studies denete—modeling -activities through 
hybrid approaches using ontologies and ML approaches are shown in 
Table III. These methods can model and reason concurrent and 
interleaved activities in both single-user and multi-user environments—, 
while tke—only single-user approaches te—could model and reason with 
uncertainty. However, while Ontelegies-ontologies are-offer the best tools 
for expressiveness, they do not capture uncertainty. To address this 
situation, probabilistic knowledge is used in these systems. Fhe 
¥Researchers has—have used Markov models, fuzzy sets, and Dempster 
Shafer theory to deal with uncertainty in a-single-user environments for 
Sequentialsequential, Mterleavedinterleaved, and GConeurrentconcurrent 
activities. 


IV. RESEARCH CHALLENGES AND FEFUTURE DBIRECTIONS 


The existing techniques and systems have significant limitations to 
recognize composite activities for multi-user i»—real-time systems, 
especially when data is coming from various sources and is incomplete 
and/or contradictory. 


Firstly, the existing approaches can model only simple or complex 
activities, and they fail to consider multiple and composite activities in 
real-time environments [43]4n-realtime-envirenments, such Fa a person 
with dementia living with his-a partner, alongside whom-and- they deing 
undertake activities in parallel and collaboratively. 


Secondly, the home context can be dynamic if more than one person 
lives in the home, or if the person has frequent visitors. Older people are 
often have alse-knewn—te—have—pets which ¢that-may roam around the 
home. This dynamic environment can make it challenging to detect and 
recognize the sensor-captured ADLs due to their unstructured and 
complex nature along with the number of individuals, infinite variations 
in their speed, and style of performance [48]. 


Thirdly, the sensor data is coming from different sources in a 
particular time frame, which induces more is-uncertainty into automated 
monitoring systems. To measure the uncertainty of events using 


knowledge-based reasoning (i.e.-9, ontology) would not produce accurate 
results [24], because ontologies are not ideal to represent uncertainties, 
because their specifications of a domain are strict and well-defined [82]. 


Moreover, due to age-related changes in health and the worsening of 
chronic disease, the quality of life of older adults may fluctuate and 
decline over time, with periods of stabilization. Monitoring ADLs, and 
functional decline and frailty is important so that the older adult living in 
the smart home can receive supportive care early on to prevent adverse 
health outcomes whenever possible [83]. 


In summary, based on a critical analysis of the existing literature, the 
following research challenges have been identified: 


e Most of the existing systems are limited to a single user in a home 
performing ADLs. Some research attempts have been attempted to 
perform ADL recognition in multi-users in a smart home 
environment, hewever—but ADL composite activities have not 
generally been considered, and certainly not commensurately with 


their obvious prevalence in real applications. 
e Single-user environment models are incomplete and contradictory, 


hewever—but Muttimulti-user user environment research is not able 
to model incomplete and/or contradictory knowledge. For instance, 
they lack the ability to capture uncertain information in terms of 
contradictory events—_(e.g., differing healthcare preferences of 
among professionals concerning treatment options)ene—decter—on 


e Very few attempts have been made to relate ADL-acthitiesADLs 
with health events, —such as recognizing changes in cognitive 
function or worsening of cognition in older adults with 
Dementiadementia. Potential signs of worsening dementia may 
include changes in balance and gait or ruminating in the same 
room, and changes in the performance of composite activities (i.e., 
attempting to prepare the meal in the bedroom). Moreover, all 
previous attempts were limited to a single-user environment. 


To overcome these limitations, there is a need to design and develop 
rules and ontologies to model composite activities in a real-time system 
for the ADLs environment for multi-users. For modeling and reasoning 
ADLs, the system weuld-should be able to understand what happens in 
terms of contradictory events coming from various sources, integrate 
them, and reason about events. To handle incomplete/or contradictory 
information in real-time coming from various sources, an argumentation 
approach will be used to capture and reason with ADL-actritiesADLs 
with the CEP engine, which processes multiple streams of data and 
reason about them. Also, it is essential to keep in mind that the most 
important priority is to wit-provide a real-time monitoring system to alert 
patients, caregivers, and doctors about cognitive behavior changes for 
patients. 


V. CONCLUSION 


Activity is a pivotal task for the identification of ADLs in the smart 
home environment. To detect and monitor ADLs and assist individuals to 
secure their safety and well-being, different types of activities such as 
atomic, complex, and composite are-need to be investigated by-in detail. 
researchers._therefere—lIthe smart home environment provides advanced 
monitoring by making use of technology, where clinicians have worked 
with engineers and computer scientists to obtain clinically relevant 
ground truth to model potential health events. 


This paper provides a systematic review regarding ADLs in the smart 
home environment and hightghted—highlights challenges in terms of 
composite activities in a multi-user environment, especially conflicting 
data in real-time. Fhereis—aA number of solutions that-have—-been—are 
suggested previdecd-in terms of models, applications, and architectures, 
which we categorized into three broad categories-ie—: Learninglearning- 
based-appreaches, knowledge-based-appreaches, and hybrid approaches. 
This survey paper provides a roadmap to identify limitations in past and 
work in the future more effectively. 


